Introduction to WDI, Part I

1 Datascience for Macroeconomics using WDI

1.1 Introduction

1.1.1 Datascience

Datascience is an interdisciplinary academic field to extract or extrapolate knowledge and insights from noisy, structured, and unstructured data.

Why datascience? There are many reasons, but I believe it would promote evidence based, non-biased thinking.

1.1.1.1 Exploratory Data Analysis, EDA

EDA is an iterative cycle that helps you understand what your data says. When you do EDA, you:

1. Generate questions about your data

2. Search for answers by visualising, transforming, and/or modeling your data

3. Use what you learn to refine your questions and/or generate new questions

EDA is an important part of any data analysis. You can use EDA to make discoveries about the world; or you can use EDA to ensure the quality of your data, asking questions about whether the data meets your standards or not. (Posit Primers: EDA)

1.1.2 Macroeconomics

The study of the entire economy of a region, a country, or the entire world

1.1.3 World Development Indicators

World Development Indicators (WDI) is the World Bank’s premier compilation of international statistics on global development. Drawing from officially recognized sources and including national, regional, and global estimates, the WDI provides access to approximately 1,600 indicators for 217 economies, with some time series extending back more than 50 years. The database helps users find information related to development, both current and historical. The topics covered in the WDI range from poverty, health, and demographics to GDP, trade, and the environment.

The World Development Indicators website provides access to data as well as information about data coverage, curation, and methodologies, and allows users to discover what type of indicators are available, how they are collected, and how they can be visualized to analyze development trends. (Wikipedia: Link)

1.2 Open Data and Public Data

Datascience and Artificial Intelligence

• Computer Resources

• Data

  • Big Data

  • Open and Public Data

Q. Explain ‘Open Data’ and ‘Public Data’, and their differences.

A. AI Poe Assistant. (Poe.com)

“Open data” and “public data” are related concepts that refer to different aspects of data accessibility and usage.

Open Data: Open data refers to the idea that certain data should be freely available to the public, without restrictions on access, usage, or redistribution. Open data is typically released in a machine-readable format, such as CSV, JSON, or XML, and is accompanied by an open license that allows anyone to use, reuse, and distribute the data for any purpose. The goal of open data is to promote transparency, collaboration, and innovation by enabling individuals, organizations, and communities to analyze and build upon the data.

Open data often includes government data, scientific research data, public records, and other datasets that are deemed to be of public interest. Governments, international organizations, and various institutions actively promote the release of open data to foster accountability, drive economic growth, and empower citizens to make informed decisions.

Public Data: Public data, on the other hand, refers to any data that is accessible or available to the public. It is a broader term that encompasses both open data and other types of data that may have certain restrictions on access or usage. Public data can include information that is freely and openly available, as well as data that is accessible through specific channels, such as public libraries, government agencies, or subscription-based services.

Public data can include a wide range of information, such as government reports, census data, legal documents, financial statements, and more. While public data may be accessible to the public, it does not necessarily guarantee the same level of openness, usability, or freedom as open data. Public data may have limitations on its usage, copyright restrictions, or conditions imposed by the data provider.

In summary, open data specifically refers to data that is freely accessible, usable, and redistributable without restrictions, often accompanied by an open license. Public data, on the other hand, is a broader term that includes both open data and other data that may have varying degrees of accessibility and usage restrictions.

1.2.0.1 World Bank on Open Data for Open Government

Visit Open Government Data Toolkit.

1.3 R on RStudio

1.3.1 Install R and RStudio

• R: https://www.r-project.org
• RStudio: https://posit.co

1.3.2 Posit Cloud

• Posit.cloud: https://posit.cloud/

  • Posit Primers: interactive tutorials

  • Cheatsheets: RStudio IDE, Data Visualization, Data Transformation, Data Import, RMarkdown, etc.

• Shared Project: https://posit.cloud/content/5539763

• Create Your Copy and Download it.

1.4 Datascience Workflow

The image above is from R4DS(2e) by Hadley Wickham, Mine Çetinkaya-Rundel and Garrett Grolemund

1.5 First Datascience

We use two packages, tidyverse and WDI.

R packages are extensions to the R statistical programming language. R packages contain code, data, and documentation in a standardized collection format that can be installed by users of R, typically via a centralized software repository such as CRAN (the Comprehensive R Archive Network). [CRAN Link]

• tidyverse: The tidyverse is a collection of open source packages for the R programming language introduced by Hadley Wickham and his team that “share an underlying design philosophy, grammar, and data structures” of tidy data. Characteristic features of tidyverse packages include extensive use of non-standard evaluation and encouraging piping. [CRAN Link]

• WDI: Search and download data from over 40 databases hosted by the World Bank, including the World Development Indicators (‘WDI’), International Debt Statistics, Doing Business, Human Capital Index, and Sub-national Poverty indicators. [CRAN Link]

1.5.1 Setup

Step 1. Install packages if necessary.

install.packages("tidyverse")
install.packages("WDI")

Step 2. Load packages.

library(tidyverse)

── Attaching core tidyverse packages ───────────────────────────────────────────────────────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.3     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.0
✔ ggplot2   3.4.3     ✔ tibble    3.2.1
✔ lubridate 1.9.2     ✔ tidyr     1.3.0
✔ purrr     1.0.2     ── Conflicts ─────────────────────────────────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the ]8;;http://conflicted.r-lib.org/conflicted package]8;; to force all conflicts to become errors

library(WDI)

Step 3. Create a data directory for the first time.

dir.create("data")

Step 4. Set ‘System Language’ to be English, recommended.

Sys.setenv(LANG = "en")

1.5.2 Importing GDP Data

The following code chunk is to download GDP data with the following indicator code.

WDI indicator: NY.GDP.MKTP.PP.KD

df_gdp <- WDI(indicator = "NY.GDP.MKTP.PP.KD")

N.B. There are many GDP related data in WDI, for example, “NY.GDP.MKTP.CD”

To avoid the internet traffic, save the data and reuse it.

CSV: comma separated values, a text format of a data.

write_csv(df_gdp, "data/gdp.csv")

Run codes above only once to download and write the data into the data directory.

df_gdp <- read_csv("data/gdp.csv")

Rows: 16758 Columns: 5── Column specification ─────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr (3): country, iso2c, iso3c
dbl (2): year, NY.GDP.MKTP.PP.KD
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

1.5.3 Viewing Data

First, look at the data by head, str ucture, and summary.

head: display the first 6 rows by default

head(df_gdp)

2.561800e+12 is in scientific notation, i.e., \(2.561800 \times10^{12} = 2,562,800,000,000\).

str: display the structure of an object

str(df_gdp)

spc_tbl_ [16,758 × 5] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
 $ country          : chr [1:16758] "Africa Eastern and Southern" "Africa Eastern and Southern" "Africa Eastern and Southern" "Africa Eastern and Southern" ...
 $ iso2c            : chr [1:16758] "ZH" "ZH" "ZH" "ZH" ...
 $ iso3c            : chr [1:16758] "AFE" "AFE" "AFE" "AFE" ...
 $ year             : num [1:16758] 2022 2021 2020 2019 2018 ...
 $ NY.GDP.MKTP.PP.KD: num [1:16758] 2.56e+12 2.47e+12 2.37e+12 2.43e+12 2.38e+12 ...
 - attr(*, "spec")=
  .. cols(
  ..   country = col_character(),
  ..   iso2c = col_character(),
  ..   iso3c = col_character(),
  ..   year = col_double(),
  ..   NY.GDP.MKTP.PP.KD = col_double()
  .. )
 - attr(*, "problems")=<externalptr> 

summary: display the summary of an object

summary(df_gdp)

   country             iso2c              iso3c                year      NY.GDP.MKTP.PP.KD  
 Length:16758       Length:16758       Length:16758       Min.   :1960   Min.   :2.482e+07  
 Class :character   Class :character   Class :character   1st Qu.:1975   1st Qu.:1.824e+10  
 Mode  :character   Mode  :character   Mode  :character   Median :1991   Median :1.055e+11  
                                                          Mean   :1991   Mean   :3.329e+12  
                                                          3rd Qu.:2007   3rd Qu.:1.083e+12  
                                                          Max.   :2022   Max.   :1.390e+14  
                                                                         NA's   :9096       

In RNotebook, the following also displays the first 1000 rows of the data in the paged format.

df_gdp

1.5.4 Transformation

|> , or %>%, is called a pipe operator and df_gdp |> filter(country == COUNTRY) is same as

filter(df_gdp, country == COUNTRY) .

filter : Keep rows that match a condition

COUNTRY <- "Japan"
df_gdp |> filter(country == COUNTRY)

1.5.5 Visualization

1.5.5.1 GDP of Japan

ggplot() + geom_line(): A tidyverse function of draw a line graph

aes(year, NY.GDP.MKTP.PP.KD) : aesthetic mapping sending year to x-axis and NY.GDP.MKTP.PP.KD to y-axis

COUNTRY <- "Japan"
df_gdp |> filter(country == COUNTRY) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD)) + geom_line()

Let’s delete the rows with missing values using drop_na(NY.GDP.MKTP.PP.KD), a transformation.

COUNTRY <- "Japan"
df_gdp |> filter(country == COUNTRY) |> drop_na(NY.GDP.MKTP.PP.KD) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD)) + geom_line()

1.5.5.2 GDP of the World

COUNTRY <- "World"
df_gdp |> filter(country == COUNTRY) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD)) + geom_line()

1.5.5.3 Exercise.

1. Delete the missing data of the GDP of the World by mimicking the one of Japan.
2. Write down your observations and questions.

COUNTRY <- "World"
df_gdp |> filter(country == COUNTRY) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD)) + geom_line()

Observations and Questions

• e.g. The GDP of the world is continuously increasing since 1990.

1.5.5.4 GDP of the countries

By country names

COUNTRIES <- c("Japan", "China", "India", "United Kingdom", "United States", "Germany", "France")
df_gdp |> filter(country %in% COUNTRIES) |> drop_na(NY.GDP.MKTP.PP.KD) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD, color = country)) + geom_line()

ISO2C <- c("JP", "CN", "ID", "UK", "US", "DE", "FR")
df_gdp |> filter(iso2c %in% ISO2C) |> drop_na(NY.GDP.MKTP.PP.KD) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD, color = iso2c)) + geom_line()

1.5.5.5 Exercise.

What happens if you replace color = iso2c at the bottom of the code above with colour = iso2c , color = country , col = country ?

ISO2C <- c("JP", "CN", "ID", "UK", "US", "DE", "FR")
df_gdp |> filter(iso2c %in% ISO2C) |> drop_na(NY.GDP.MKTP.PP.KD) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD, color = iso2c)) + geom_line()

1.5.5.6 List of Countries and iso2c

(df_codes <- df_gdp |> distinct(country, iso2c))

1.5.5.7 Exercise.

Set COUNTRIES and/or ISO2C to draw line graphs of GDP.

COUNTRIES <- c() # surround the country name with quotation marks, and use a comma as a separator
df_gdp |> filter(country %in% COUNTRIES) |> drop_na(NY.GDP.MKTP.PP.KD) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD, color = country)) + geom_line()

ISO2C <- c() # surround the iso2c code with quotation marks, and use a comma as a separator
df_gdp |> filter(iso2c %in% ISO2C) |> drop_na(NY.GDP.MKTP.PP.KD) |>
  ggplot(aes(year, NY.GDP.MKTP.PP.KD, color = iso2c)) + geom_line()

2 World Development Indicators

2.1 Find Indicator Codes

1. World Bank Home Page

   • World Development Indicators: Link

   • World Bank Open Data: Link

   • WDI Statistical Tables: Link

2. Excel Files

   • Data Catalogues: Link Download Excel File

3. API Search

WDIsearch(string = "gdp", field = "name")

WDIsearch(string = "NY.GDP.MKTP.PP.KD", field = "indicator", short = FALSE)

2.1.0.1 Assignment

1. Find at least one WDI indicator with its name and its code.

2. Find at least one pair of WDI indicators with their names and their codes you want to study their relation.

2.2 GDP and GDP per Capita

1. GDP, PPP (constant 2017 international $): NY.GDP.MKTP.PP.KD

2. Population, total: SP.POP.TOTL

3. Calculate GDP per Capita

   • GDP per capita, PPP (constant 2017 international $): NY.GDP.PCAP.PP.KD

• GDP, PPP (constant 2017 international $) PPP GDP is gross domestic product converted to international dollars using purchasing power parity rates. An international dollar has the same purchasing power over GDP as the U.S. dollar has in the United States. GDP is the sum of gross value added by all resident producers in the country plus any product taxes and minus any subsidies not included in the value of the products. It is calculated without making deductions for depreciation of fabricated assets or for depletion and degradation of natural resources. Data are in constant 2017 international dollars. ID: NY.GDP.MKTP.PP.KD

• Population, total Total population is based on the de facto definition of population, which counts all residents regardless of legal status or citizenship. The values shown are midyear estimates. ID: SP.POP.TOTL

df_gdppcap <- WDI(indicator = c(gdp = "NY.GDP.MKTP.PP.KD", pop = "SP.POP.TOTL", gdppcap = "NY.GDP.PCAP.PP.KD"), extra = TRUE)

write_csv(df_gdppcap, "data/gdppcap.csv")

df_gdppcap <- read_csv("data/gdppcap.csv")

Rows: 16758 Columns: 15── Column specification ─────────────────────────────────────────────────────────────────────────────────────────────────────────
Delimiter: ","
chr  (7): country, iso2c, iso3c, region, capital, income, lending
dbl  (6): year, gdp, pop, gdppcap, longitude, latitude
lgl  (1): status
date (1): lastupdated
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

str(df_gdppcap)

spc_tbl_ [16,758 × 15] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
 $ country    : chr [1:16758] "Afghanistan" "Afghanistan" "Afghanistan" "Afghanistan" ...
 $ iso2c      : chr [1:16758] "AF" "AF" "AF" "AF" ...
 $ iso3c      : chr [1:16758] "AFG" "AFG" "AFG" "AFG" ...
 $ year       : num [1:16758] 2014 2012 2009 2013 1971 ...
 $ status     : logi [1:16758] NA NA NA NA NA NA ...
 $ lastupdated: Date[1:16758], format: "2023-09-19" "2023-09-19" "2023-09-19" "2023-09-19" ...
 $ gdp        : num [1:16758] 7.02e+10 6.47e+10 4.99e+10 6.83e+10 NA ...
 $ pop        : num [1:16758] 32716210 30466479 27385307 31541209 11015857 ...
 $ gdppcap    : num [1:16758] 2144 2123 1824 2165 NA ...
 $ region     : chr [1:16758] "South Asia" "South Asia" "South Asia" "South Asia" ...
 $ capital    : chr [1:16758] "Kabul" "Kabul" "Kabul" "Kabul" ...
 $ longitude  : num [1:16758] 69.2 69.2 69.2 69.2 69.2 ...
 $ latitude   : num [1:16758] 34.5 34.5 34.5 34.5 34.5 ...
 $ income     : chr [1:16758] "Low income" "Low income" "Low income" "Low income" ...
 $ lending    : chr [1:16758] "IDA" "IDA" "IDA" "IDA" ...
 - attr(*, "spec")=
  .. cols(
  ..   country = col_character(),
  ..   iso2c = col_character(),
  ..   iso3c = col_character(),
  ..   year = col_double(),
  ..   status = col_logical(),
  ..   lastupdated = col_date(format = ""),
  ..   gdp = col_double(),
  ..   pop = col_double(),
  ..   gdppcap = col_double(),
  ..   region = col_character(),
  ..   capital = col_character(),
  ..   longitude = col_double(),
  ..   latitude = col_double(),
  ..   income = col_character(),
  ..   lending = col_character()
  .. )
 - attr(*, "problems")=<externalptr> 

df_gdppcap |> select(region, income, lending) |> lapply(unique)

$region
[1] "South Asia"                 "Aggregates"                 "Europe & Central Asia"      "Middle East & North Africa"
[5] "East Asia & Pacific"        "Sub-Saharan Africa"         "Latin America & Caribbean"  "North America"             
[9] NA                          

$income
[1] "Low income"          "Aggregates"          "Upper middle income" "Lower middle income" "High income"        
[6] NA                    "Not classified"     

$lending
[1] "IDA"            "Aggregates"     "IBRD"           "Not classified" "Blend"          NA              

COUNTRY <- "World"
df_gdppcap |> filter(country == COUNTRY) |>
  ggplot(aes(year, gdppcap)) + geom_line()

COUNTRY <- "World"
df_gdppcap |> filter(country == COUNTRY) |>
  ggplot(aes(year, pop)) + geom_line()

2.2.0.1 Exercise.

Write your observations and questions.

2.2.1 GDP Per Capita

df_gdppcap2 <- df_gdppcap |> drop_na(pop) |> 
  mutate(PCAP = gdp/pop, .after = gdppcap)

df_gdppcap2

2.2.1.1 Check against GDP per capita, PPP

df_gdppcap2 |> drop_na(gdppcap, PCAP) |> mutate(near = near(gdppcap, PCAP)) |> 
  summarize(numberofdata = n(), sum(near))

df_gdppcap2 |> filter(!near(gdppcap, PCAP))

2.2.1.2 Exercise.

Write your observations and questions.

2.2.2 Visualization

Two useful questions.

1. What type of variation occurs within my variables?

2. What type of covariation occurs between my variables?

See Link.

2.2.2.1 Ranks.

arrange(desc(gdp)) is to reorder in descending order of gdp, arrange(gdp) in ascending order.

df_gdppcap |> filter(year == 2022, region != "Aggregates") |> 
  drop_na(gdp) |> arrange(desc(gdp))

2.2.2.2 Exercises.

1. Find the top 10 of the countries with the highest GDP per capita.

2. Find the top 10 of the countries with the lowest GDP per capita.

3. Find the top 10 of the countries with the largest population.

4. Find the top 10 of the countries with the smallest population.

2.2.2.3 Scatter Plot

What type of covariation occurs between my variables?

df_gdppcap2 |> filter(year == 2022, region !="Aggregates") |> 
  drop_na(gdp, pop) |> 
  ggplot(aes(pop, gdp)) + geom_point()

df_gdppcap2 |> filter(year == 2022, region !="Aggregates") |> 
  drop_na(gdp, pop) |> 
  ggplot(aes(pop, gdp)) + geom_point() + 
  scale_x_log10() + scale_y_log10()

df_gdppcap2 |> filter(year == 2022, region !="Aggregates") |> 
  drop_na(gdp, pop) |> 
  ggplot(aes(pop, gdp)) + geom_point() + 
  geom_smooth(method = "lm", se = FALSE) +
  scale_x_log10() + scale_y_log10()

df_gdppcap2 |> filter(year == 2020, region !="Aggregates") |> drop_na(gdp, pop) |> 
  ggplot(aes(pop, gdp, color = region)) + geom_point() + 
  scale_x_log10() + scale_y_log10()

df_gdppcap2 |> filter(year == 2020, region !="Aggregates") |> 
  drop_na(gdp, pop) |> 
  ggplot(aes(pop, gdp, color = region, shape = income)) + geom_point() + 
  scale_x_log10() + scale_y_log10()

df_gdppcap2 |> filter(year == 2020, region !="Aggregates") |> 
  drop_na(gdp, gdppcap, pop) |> 
  ggplot(aes(gdppcap, gdp, color = region, size = pop)) + geom_point() + 
  scale_x_log10() + scale_y_log10()

install.packages("plotly")

library(plotly)

Registered S3 method overwritten by 'data.table':
  method           from
  print.data.table     
Registered S3 method overwritten by 'htmlwidgets':
  method           from         
  print.htmlwidget tools:rstudio

Attaching package: ‘plotly’

The following object is masked from ‘package:ggplot2’:

    last_plot

The following object is masked from ‘package:stats’:

    filter

The following object is masked from ‘package:graphics’:

    layout

test <- df_gdppcap2 |> filter(year == 2020, region !="Aggregates") |> drop_na(gdp, pop) |> 
  ggplot(aes(color = country, shape = region, pop, gdp)) + geom_point() + 
  scale_x_log10() + scale_y_log10() + theme(legend.position = "none")
test |> ggplotly()

Warning: The shape palette can deal with a maximum of 6 discrete values because more than 6 becomes difficult to
discriminate; you have 7. Consider specifying shapes manually if you must have them.